Module manager: Dr. Mai Elshehaly
Email: M.H.Elshehaly@leeds.ac.uk
Taught: Semester 1 (Sep to Jan) View Timetable
Year running 2019/20
We will assume a good standard of C++ programming, including use of classes, basic templates, and overloading. Knowledge of computer architecture, in particular the memory hierarchy.
This module is not approved as a discovery module
This module introduces students to the principles and practice of parallel and concurrent programming on shared memory architectures (both CPU and GPU). It covers the fundamental concepts underlying concurrency, in particular the complexity of managing shared resources and the language/data abstractions used to mediate interaction between threads of execution. The module also introduces students to the challenge of designing algorithms that minimise synchronisation bottlenecks, and the use of tools to understand and tune the performance of code running on parallel architectures.
* understand the fundamental concepts of concurrent programming, including: threads, shared variables, critical regions, atomics, semaphores, race conditions, deadlock
* survey and critique examples of parallel architecture, covering both multi-core (CPU) and many-core (GPU) platforms.
* develop the practical understanding and skills needed to design, program, test, debug, and tune non-trivial parallel and concurrent applications using industry-standard tools and APIs, eg. CUDA, C++.
* for concurrent programming, introduce students to lock-based and lock-free data structures
* for parallel programming, understand the complex relationships between data organisation, memory system and performance on parallel architectures, and the design and implementation of algorithms that exploit the architecture effectively, e.g. by reducing or eliminating use of locking.
Students will understand the main concepts underlying parallel & concurrent programming, the relationship of the concepts to both hardware and software, and the practical implications for software development. They will be able to write programs of intermediate complexity on both multi-core and many-core architectures using languages and tools that are in widespread industrial use. And they will understand the implications of concurrency and parallelism across the software development process, from the design of efficient data representations through to the challenge of testing, debugging and tuning programs. Over the course of the module students will also have further exposure to the tool ecosystem that is essential to professional software development.
* programming & development with industry-standard tools eg. C++ and CUDA
* professional software development
* The evolution of computer architecture
* Architectural examples
* Threads
* Shared variables, race conditions, deadlock, and semaphores
* Atomic types and operations
* SIMD instructions and maths
* Mutexes and locking
* Testing, debugging, and profiling multithreaded code
* Lock-based and lock-free concurrent data structures
* Thread management and pools
* Programming for the GPU: SIMT, blocks, grids, warps, kernels, and synchronization points.
* Efficient parallel programming: avoiding locking, exploiting specialised memory regions and coalesced access.
* Synchronizing concurrent operations
* Efficient CUDA programming: avoiding divergence, exploiting coalesced memory access, specialised memory regions, and streams.
* Thrust library
| Delivery type | Number | Length hours | Student hours |
|---|---|---|---|
| Class tests, exams and assessment | 2 | 22 | 44 |
| Lecture | 20 | 1 | 20 |
| Practical | 10 | 2 | 20 |
| Private study hours | 66 | ||
| Total Contact hours | 84 | ||
| Total hours (100hr per 10 credits) | 150 | ||
Parallel/concurrent programming introduces students to new hardware and software concepts that are both complex in themselves, and significantly complicate the task of programming. Of the ~6 hours per week of private study, 2 hours should be dedicated to comprehending the underlying theory through studying material from the module’s reading list. The remaining time will be spent carrying out practical programming exercises, both practical sheets and further exercises from the literature as indicated by the module leader.
The weekly supervised practical classes will provide feedback on students’ ability to deploy the new concepts covered in that week.
| Assessment type | Notes | % of formal assessment |
|---|---|---|
| Assignment | Programming Task | 20 |
| Assignment | Programming Task | 20 |
| Total percentage (Assessment Coursework) | 40 | |
Normally resits will be assessed by the same methodology as the first attempt, unless otherwise stated
| Exam type | Exam duration | % of formal assessment |
|---|---|---|
| Unseen exam | 2.0 Hrs 0 Mins | 60 |
| Total percentage (Assessment Exams) | 60 | |
Normally resits will be assessed by the same methodology as the first attempt, unless otherwise stated
Check the module area in Minerva for your reading list
Last updated: 30/09/2019
Errors, omissions, failed links etc should be notified to the Catalogue Team